1. Field of the Invention
The present invention relates to the field of producing closed captioning for television program viewers who are deaf or hard-of-hearing, and more particularly to a method and apparatus for providing closed captioning in two languages, the first language using the Roman alphabet and the second language using a syllabic alphabet, as will be more particularly defined herein.
2. Description of Related Art
Closed captioning is the method by which the audio portion of a television program or home video is processed into recorded words that can be displayed on a television screen with the aid of circuitry contained in an external device known as a captioning decoder or related decoder circuitry built into a television receiver. One known decoder commercially available through the National Captioning Institute of Falls Church, Va. is the TeleCaption (TM) 4000 decoder.
The purpose of the decoder is to receive encoded captioning data from a data transmission means, for example from a television data stream of a high definition television signal or from data transmitted at a particular line of the vertical blanking interval of a standard NTSC, PAL, SECAM, etc. television signal which may be used for data transmission. This line in the standard television signal may be any of lines 10 through 21 of the vertical blanking interval of a known standard television signal format adapted for use in the United States. In accordance with United States standard National Television SubCommittee (NTSC) format as described at 47 Code of Federal Regulations (CFR) Section 73.682 (a)(22), the chosen data transmission line is line 21, which is immediately adjacent to active video. In one higher resolution European standard television broadcast signal, the chosen line is line 16 or line 329 of the vertical blanking interval, in another, line 22 or line 335.
Particular details of specifications for a decoder have been published by the Federal Communications Commission on Jun. 13, 1991 at 47 CFR section 15.119 through the Federal Register, to become effective Jul. 15, 1991. There is provided a transmission and encoding scheme for transmitting special characters such as musical notes, the Roman alphabet, punctuation, attributes, control codes and such in hexadecimal format. The special character set further includes alphabet characters, for example, having a grave accent so that European languages such as French or Spanish may be transmitted. The decoder decodes the captioning and control code data transmitted on line 21 in accordance with the published format. The decoder includes memory for looking up a stored character set and causes a display of closed captioned text on a television screen simultaneously with the image of the person speaking the text.
An overall block schematic diagram for the TeleCaption 4000 decoder is shown as prior art FIG. 1A. A broadcast television signal is received at a television tuner 101 selectively actuated by a controller 103, preferably a microprocessor, to tune to a selected television channel. In particular, controller 103 may be a Motorola 6805P6 microprocessor, part no. F24K189 or comparable microprocessor known in the art. The tuner 101 is actuated under user control, preferably by channel up and channel down key or other input device input (not shown). The received television signal is demodulated from the selected channel frequency and the audio and video carriers at baseband separated at the television tuner 101. The audio signal is provided to a volume control circuit 102 and the video to a close-caption processor 104, both under control of the controller circuit 103. Audio preferably may be regulated under user control via a user input device (not shown), for example, via up and down volume keys. Regulated audio at a desired volume level is provided from the volume control circuit 102 to a channel 3/4 modulator 105. At the same time, the close-caption processor 104 extracts the textual and command control data including caption data representative of the audio transmission and outputs an overlaid video signal including captions to the channel 3/4 modulator 105. The close-caption processor 104 most conveniently comprises a Sierra NC74021 application specific integrated circuit, available from the National Captioning Institute, and a microprocessor, for example, a Motorola 6803 custom mask SC93193P controller, also available from the National Captioning Institute. Further details of one closed caption processor comprising these two integrated circuits are provided in FIG. 1B. The channel 3/4 modulator 105, preset to one or the other of channels 3 or 4, modulates the combined video and audio signal to the selected frequency. The output of the channel 3/4 modulator 105 is then provided to an associated television set (not shown), which if tuned to the preset channel 3 or 4, will display the output of the caption decoder of FIG. 1A.
A schematic circuit diagram of the close-caption processor 104 or decoder section of FIG. 1A is represented as FIG. 1B. The received baseband video signal is provided to a close-caption overlay circuit 112, to a close-caption timing circuit 106 and to a data extractor 107. The close-caption timing circuit 106 regards the input video signal to determine the location of the vertical blanking interval, the identity of the received field, be it odd or even, and determines the starting point of the selected line, for example, line 21 for the identified field. The close-caption timing circuit 106, operating at sub-line frequency, also generally clocks all the operations of the principal building block circuits of the decoder circuitry. For example, the close-caption timing circuit 106 clocks and signals the data extractor circuit 107 to extract the data received at the selected line.
Referring briefly to FIG. 2A, typically, a clock run in burst is used to obtain a half power value for data slicing and the data extractor 107, clocked by the timing circuit 106 extracts a serial data stream comprising two data bytes at a time and forwards the extracted data to controller 109. At the same time, the close-caption timing circuit 106 clocks and signals the attribute register circuit 108 when to begin to store attribute data for an output caption to be generated and eventually overlaid on the incoming video signal via the close-caption overlay circuit 112. The close-caption timing circuit 106 also clocks and signals the close-caption controller 109, which most conveniently comprises a microprocessor, to accept data from the data extractor circuit 107. Also, the close-caption timing circuit 106 clocks the display RAM circuit 110 as it receives temporary data from the close caption controller 109, outputs attribute data to the attribute register and pointer data to a character ROM 111, also clocked by the close-caption timing circuit 106.
Pointer data output from the display RAM 110 points uniquely to a permanently stored character set stored in the character ROM circuit 111. The attribute register circuit 108 further provides attribute data to the character ROM 111 for characters. In a caption mode of operation, attributes, for example, may be related to style, whether or not the character is to blink or to appear in a particular color. Attribute data provided to the close-caption overlay circuit 112 relates to attributes for an entire overlay or caption. In this manner, an entire caption is generated for display on an associated television set which may be rolled-up (scrolled), popped-on (displayed all at one time), or painted-on (displayed in a continuously printing manner).
If a caption is to be printed in italics, the closed caption timing circuit 106 is signalled by the closed caption controller responsive, for example, to receipt of a mid-row italics code to output timing signals to generate pointer data output from display RAM 110. However, closed caption timing circuit 106 outputs clocking signals at a different rate to character ROM 111 in such manner that the output character data of character ROM 111 is intentionally slanted to appear in italics.
An application specific integrated circuit (ASIC) for performing the decoding of teletext signals transmitted with a television signal is available from ITT Corporation and is commercially known as the Teletext Processing Unit (TPU) 2740, a 24 pin ASIC. This chip also generates English teletext characters for display as a teletext television image, typically a static image of textual data. Teletext is a service provided, for example, over broadcast television channels, cable television channels or through direct broadcast satellite systems, and while not related specifically to closed captioning, relates to a method of providing textual information which may be superimposed on an action image or a plain background.
Data required for closed captioning requires both textual or captioning data and display command data, for example, for synchronizing the caption data with the spoken dialogue of a television program. Thus, a caption decoder includes circuitry which additionally decodes display command data to display a caption at the bottom of a television image.
Captioned television thus enables viewers to read the dialogue and narration of programs. Closed captioning provides access to the entertainment, educational and informational benefits of television for viewers who are deaf or hard-of-hearing. When the spoken word is in a foreign language and sub-titles in a native tongue are provided, people learning the foreign language can enjoy the benefits of such captioned television as well.
Known decoders generally comprise a character generator as per FIG. 1B or similar circuitry which causes captions to appear in the lower portion of the television screen, below the video image of the speaker. As already described, the character generator may also be built into an associated television set. The video information appearing on the horizontal lines of the caption image are appropriately replaced with character data output by the character generator, for example, via closed caption overlay circuit 112. Character sizes vary proportionately to the size of the television screen. On a nineteen inch screen, for instance, captions appear to be about one half inch high. The captions are easily visible--typically white letters against a black background. Appearing at the bottom of the picture, captions typically do not obstruct one's view of the action portion of the video program image, that is, that portion of the image where the normal viewer, without captioning, would focus their attention in order to enjoy the displayed image.
Closed captioning technology was developed by the Public Broadcasting Service (PBS) during the period 1973-1979. Numerous over-the-air test transmissions were conducted to test all aspects of the caption generation including encoding, decoding, and the display of captions. Service was launched in March of 1980 by the then newly created National Captioning Institute in cooperation with the television networks. The original target audience for captioning included people who were deaf or hard-of-hearing (about 20 million people), but now includes people learning English as a second language and those learning to read, such as students with learning disabilities. Weekly availability in the United States of captioned programs now exceeds 400 hours between network TV, cable television and syndicated programs.
For encoding according to NTSC and Federal Communications Commissions Standards, broadcasters may obtain a model EN230 or smart encoder manufactured by E.E.G. Enterprises, Inc. of Farmingdale, N.Y. This same company manufactures a model EN210 or simple encoder as well. The encoder inserts caption data into fields one and two at line 21 of the vertical blanking interval. A so-called smart encoder receives data to be inserted through an RS232 serial data port. The smart encoder, having a built-in decoder, can recognize gaps in caption data and can fill the gaps with additional caption or text service data. For example, if already existent captioning data appears at line 21 but there exists a capacity to display up to fifteen rows of thirty-two characters, there is spare capacity for additional captions or text services. These text services may comprise emergency announcements such as tornado warnings or other messages to the target audience. The simple encoder is used to create new captions and has no internal decoder.
Developing high definition television standards also include a means for transmitting captioning data.
A means for encoding a character set for certain alphabet characters, namely, the ASCII character code is well known and comprises 96 different hexadecimal codes for pointing to corresponding characters in memory. Thus, two bytes transmitted one byte per field or two bytes per frame uniquely point to an alphabetic, control, punctuation, or other character for display in English. As the NTSC transmission rate provides for the transmission of 30 frames (of two interlaced fields) per second, the data transmission rate for alphabet characters can be as high as sixty characters per second which are stored and displayed, as controlled by display command data, simultaneously with the spoken message in intelligible form, such as in complete sentences.
A problem with existing decoders is that they are only capable of receiving captioning data in one language at a time, for example English, French, German, or Spanish, which is based on the Roman alphabet. Nevertheless, the NTSC format is used in other major countries as well such as Korea, Japan, and Taiwan. In such countries therefore, there is an opportunity to provide captioning in the languages of these countries, namely Korean, Japanese and Chinese, consistent with the NTSC standards and known methods of caption transmission utilizing the data-carrying capabilities of the vertical blanking interval.
On the other hand, languages, such as Korean, Japanese and Chinese, and for the purposes of the present application, the Korean language will be discussed as exemplary, are unique in their means of recording the spoken word. Notwithstanding a danger of oversimplification, such languages are recorded in the form of pictographic or ideographic communications, rather than on a Roman letter by letter basis as in English to form syllables from a plurality of letters. The Korean alphabetic character set, in particular, provides hints as to how to form one's mouth to speak a syllable represented by a character. Consequently, fewer characters are required to form the same thought or sentence as would be required in English, but the greater variety of characters and their relative complexity make the ability to print the Korean character set difficult.
From the Korean government publication Han'gul, (Korean Alphabet and Language), Korea Background Series Volume No. 9, available through the Korean Overseas Information Service and published in December, 1981, it is described how the Korean alphabet was developed by King Sejong in relatively modern times to phonetically describe sounds making up syllables of words. The Han'gul alphabet, a syllabic alphabet, comprises twenty-eight letters comprising seventeen consonants and eleven vowels. Consonants and vowels alternate in words to form syllables of words which phonetically describe how to pronounce Korean words.
The Korean language is thus unlike English. In the English language, strings of consonants may be run together such as "str" in "straight" and a combination of vowels and consonants may have to be interpreted together before one may pronounce a word, such as the combination of letters "aight" in "straight".
For the purposes of the present application and claims, and for want of better terminology, the term "Roman alphabet" will be used to describe a category of alphabets for Latin-based languages, for example, such languages as English, French, Spanish, German, Dutch, Swedish and such. The term "syllabic alphabet" will be used as a term of art to describe a category of alphabets used in various languages such as Korean.
According to the well-known ASCII coding format, two consecutive bytes of data may combine to form a hexadecimal pointer into a Roman alphabetic or other character memory to obtain letters, numbers, or other special characters for display. There is no well-known, published coding scheme for providing captions in a syllabic language such as the Korean language.
Associated with solving the problem of providing an electronic typewriter for the Korean language, C. H. Yeh in U.S. Pat. No. 4,187,031, suggests an encoding scheme whereby sets of hexadecimal numbers be provided for the Hangul character set, one for the alphabetic symbol, and another, called a form code, for defining how to build a Hangul character. Such a coding system, however, has its limitations if adapted for data transmission within a line of a vertical blanking interval. Yet, there is an obvious need to provide captioning for deaf or hard-of-hearing people in countries such as Korea or, more particularly, to provide dual language captioning in any country including the United States.